A Convergent Iterative Hard Thresholding for Nonnegative Sparsity Optimization

The iterative hard thresholding (IHT) algorithm is a popular greedy-type method in (linear and nonlinear) compressed sensing and sparse optimization problems. In this paper, we give an improved iterative hard thresholding algorithm for solving the nonnegative sparsity optimization (NSO) by employing the Armijo-type stepsize rule, which automatically adjusts the stepsize and support set and leads to a sufficient decrease of the objective function each iteration. Consequently, the improved IHT algorithm enjoys several convergence properties under standard assumptions. Those include the convergence to α-stationary point (also known as L-stationary point in literature if the objective function has Lipschitz gradient) and the finite identification of the true support set. We also characterize the conditions that the full sequence converges to a local minimizer of NSO and establish its linear convergence rate. Extensive numerical experiments are included to demonstrate the good performance of the proposed algorithm.